{
 "nbformat": 4,
 "nbformat_minor": 0,
 "metadata": {
  "colab": {
   "name": "Fun with tf.data.Dataset (playground).ipynb",
   "provenance": [],
   "collapsed_sections": [],
   "toc_visible": true
  },
  "environment": {
   "name": "tf22-cpu.2-2.m47",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/tf22-cpu.2-2:m47"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "89B27-TGiDNB"
   },
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "U6rgexPXmY37",
    "outputId": "cf6dce9d-5a4a-4459-c40b-4b3fc7ec8a67"
   },
   "source": [
    "import os, sys, math\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "import tensorflow as tf\n",
    "print(\"Tensorflow version \" + tf.__version__)"
   ],
   "execution_count": 1,
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Tensorflow version 2.6.0\n"
     ]
    }
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "cellView": "form",
    "id": "MPkvHdAYNt9J"
   },
   "source": [
    "#@title \"display utilities [RUN ME]\"\n",
    "def display_9_images_from_dataset(dataset):\n",
    "  plt.figure(figsize=(13,13))\n",
    "  subplot=331\n",
    "  for i, (image, label) in enumerate(dataset):\n",
    "    plt.subplot(subplot)\n",
    "    plt.axis('off')\n",
    "    plt.imshow(image.numpy().astype(np.uint8))\n",
    "    plt.title(label.numpy().decode(\"utf-8\"), fontsize=16)\n",
    "    subplot += 1\n",
    "    if i==8:\n",
    "      break\n",
    "  plt.tight_layout()\n",
    "  plt.subplots_adjust(wspace=0.1, hspace=0.1)\n",
    "  plt.show()"
   ],
   "execution_count": 2,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "w9S3uKC_iXY5"
   },
   "source": [
    "## Configuration"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "d8K6hL_kiWve"
   },
   "source": [
    "GCS_PATTERN = 'gs://flowers-public/*/*.jpg'\n",
    "CLASSES = ['daisy', 'dandelion', 'roses', 'sunflowers', 'tulips'] # flower labels (folder names in the data)"
   ],
   "execution_count": 3,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kvPXiovhi3ZZ"
   },
   "source": [
    "## Read images and labels [WORK REQUIRED]\n",
    "1. Use `fileset=`[`tf.data.Dataset.list_files`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#list_files) to scan the data folder\n",
    "1. Iterate through the dataset of filenames: `for filename in fileset:...` . \n",
    " * Does it work ? Yes, but if you print the filename you get Tensors containing strings.\n",
    " * To display the string only, you can use filename.numpy(). This works on any Tensorflow tensor.\n",
    " * tip: to limit the size of the dataset for display, you can use [`Dataset.take()`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#take). Like this: `for data in dataset.take(10): ....`\n",
    "1. Use [`tf.data.Dataset.map`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#map) to decode the JPEG files. You will find useful TF code snippets below.\n",
    " * Iterate on the image dataset. You can use .numpy().shape to only see the data sizes.\n",
    "  * Are all images of the same size ?\n",
    "1. Now create a training dataset: you have images but you also need labels:\n",
    " *  the labels (flower names) are the directory names. You will find useful TF code snippets below for parsing them.\n",
    " * If you do \"`return image, label`\" in the decoding function, you will have a Dataset of pairs (image, label).\n",
    "1. You can see the flowers and their labels with the `display_9_images_from_dataset` function. It expects the Dataset to have `(image, label)` elements."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "nwsZ8X59mu24"
   },
   "source": [
    "nb_images = len(tf.io.gfile.glob(GCS_PATTERN))\n",
    "print(\"Pattern matches {} images.\".format(nb_images))\n",
    "\n",
    "#\n",
    "# YOUR CODE GOES HERE\n",
    "#\n",
    "\n",
    "#display_9_images_from_dataset(dataset)"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZX6Vg0YZwRCP"
   },
   "source": [
    "## Useful code snippets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FszNm593wnky"
   },
   "source": [
    "### Decode a JPEG in Tensorflow"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "pmPz2WM2wTbS"
   },
   "source": [
    "def decode_jpeg(filename):\n",
    "  bits = tf.io.read_file(filename)\n",
    "  image = tf.io.decode_jpeg(bits)\n",
    "  return image"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Dax9B6W7wuxt"
   },
   "source": [
    "### Decode a JPEG and extract folder name in TF"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "kPhQl3BlxB7D"
   },
   "source": [
    "def decode_jpeg_and_label(filename):\n",
    "  bits = tf.io.read_file(filename)\n",
    "  image = tf.io.decode_jpeg(bits)\n",
    "  # parse flower name from containing directory\n",
    "  label = tf.strings.split(tf.expand_dims(filename, axis=-1), sep='/')\n",
    "  label = label.values[-2]\n",
    "  return image, label"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6UuMEBez45s5"
   },
   "source": [
    "## License\n",
    "\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "author: Martin Gorner<br>\n",
    "twitter: @martin_gorner\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "Copyright 2021 Google LLC\n",
    "\n",
    "Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "you may not use this file except in compliance with the License.\n",
    "You may obtain a copy of the License at\n",
    "\n",
    "    http://www.apache.org/licenses/LICENSE-2.0\n",
    "\n",
    "Unless required by applicable law or agreed to in writing, software\n",
    "distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "See the License for the specific language governing permissions and\n",
    "limitations under the License.\n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "This is not an official Google product but sample code provided for an educational purpose\n"
   ]
  }
 ]
}